Guillermo Garcia-Hernando Research Scientist at Niantic, Inc.

About

I'm a researcher at Niantic's AR Research team led by Gabriel Brostow and based in London, UK. Previously, I spent a couple of years as a postdoc at Imperial College London in collaboration with Samsung Research. I completed my Ph.D. in computer vision and machine learning in 2017 supervised by Tae-Kyun Kim also at Imperial. In a previous life, I was in Paris (MSc./Diplôme d'Ingénieur at Télécom Paris) and in Barcelona (BSc. at Technical University of Catalonia - UPC).

I particularly enjoy teaching, supervising students and disseminating research to audiences of different backgrounds and ages. I've been involved in the organization of academic events such as BMVC 2017, different workshops at CVPR, ICCV and ECCV conferences and also had the chance of teaching at the Nepal Winter School in AI 2018.

First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations

Guillermo Garcia-Hernando, Shanxin Yuan, Seungryul Baek, Tae-Kyun Kim. CVPR 2018.

Abstract

In this work we study the use of 3D hand poses to recognize first-person dynamic hand actions interacting with 3D objects. Towards this goal, we collected RGB-D video sequences comprised of more than 100K frames of 45 daily hand action categories, involving 26 different objects in several hand configurations. To obtain hand pose annotations, we used our own mo-cap system that automatically infers the 3D location of each of the 21 joints of a hand model via 6 magnetic sensors and inverse kinematics. Additionally, we recorded the 6D object poses and provide 3D object models for a subset of hand-object interaction sequences. To the best of our knowledge, this is the first benchmark that enables the study of first-person hand actions with the use of 3D hand poses. We present an extensive experimental evaluation of RGB-D and pose-based action recognition by 18 baselines/state-of-the-art approaches. The impact of using appearance features, poses, and their combinations are measured, and the different training/testing protocols are evaluated. Finally, we assess how ready the 3D hand pose estimation field is when hands are severely occluded by objects in egocentric views and its influence on action recognition. From the results, we see clear benefits of using hand pose as a cue for action recognition compared to other data modalities. Our dataset and experiments can be of interest to communities of 3D hand pose estimation, 6D object pose, and robotics as well as action recognition.

Paper

Dataset download

Code and instructions

Citation 

@InProceedings{FirstPersonAction_CVPR2018,
  title={First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations},
  author={Garcia-Hernando, Guillermo and Yuan, Shanxin and Baek, Seungryul and Kim, Tae-Kyun}
  booktitle = {Proceedings of Computer Vision and Pattern Recognition ({CVPR})},
  year = {2018}
}